Set generate_description to true in IndexOptions default configuration
to enable automatic description generation for improved search
functionality.

… tracking

- Remove CancelFlag and active_ops fields from Engine struct
- Delete all cancellation-related methods (cancel, reset_cancel, is_cancelled, check_cancel)
- Remove ActiveGuard struct and associated increment/decrement logic
- Eliminate cancellation checks from index, query, and related operations
- Update metrics_report documentation to reflect removed pilot decision metrics
- Clean up unused imports and dependencies

…approach

Replace the old "reasoning-native" terminology with "agentic-based" to better
describe the document engine's architecture. Remove outdated bullet points
about reasoning principles that are no longer accurate.

feat(retrieval): refactor engine to use agent-based architecture

Change the retrieval engine from using search strategies (greedy DFS, beam
search, MCTS) to an agent-based system with Orchestrator coordinating
Workers. The new system uses LLM-guided navigation commands (ls, cd, cat,
find, grep) with progress evaluation and replanning capabilities while
maintaining budget awareness.

- Update workspace package version from 0.1.30 to 0.1.31 in Cargo.toml
- Update vectorless package version from 0.1.9 to 0.1.10 in pyproject.toml